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This article discusses the use of sound for auditory information display, and in particular 
its application for exploration of scientific data, known as sonification. Sonification can 
be defined as the use of sound to display data of scientific interest in order to investígate 
structures, trends or patterns in the data. Background is provided from several perspec¬ 
tivas: the use of the senses in the history of Science, the strengths of human hearing, the 
recent technological availability of auditory interfaces, the development of sonification 
itself, and differentiation of sonification from musical practices. In Western Science, as in 
Western culture, the eye has become the predominant organ of sense; using the ear con- 
sciously in research thus implicitly questions the implications of the eye’s predominance. 
As practical examples, two applications of sonification to data-sets from the social Sciences 
are discussed in detail. We argüe that the most promising areas of application of 
sonification within the social Sciences are in the exploration of sequential data. Our two 
examples both concern sets of sequential data, one temporal, the other spatial (geographi- 
cal). Discussion of these examples is followed by consideration of the practical and cultural 
implications of working with sonification. We thus hope to further the use of sonification in 
the social Sciences, not as an alternative to visualisation or statistical approaches, but as a 
complementary tool of data analysis and exploration. 


Presenting data or information to human beings must necessarily rely on some mode of 
perceptualisation: to be perceived by the human senses, data or information must be 
rendered in suitable form. While visual rendering (visualisation) has become an everyday 
affair, the representation of data or information by sound remains much less common. 
Nonetheless, research concerned with auditory interfaces and with related issues of psy- 
choacoustics and design is a rapidly expanding field in the realm of information and com- 
munication technologies (ICT). The main concern in this emerging ñeld is to investígate 
ways of conveying information in the form of sound generated by for example a personal 
Computer. Because auditory interfaces offer alternative channels of communication, the 
ñrst beneficiarles of these research and development activities have been people with 
visión or speech impairments. When such applications are designed in a satisfactory and 
user-friendly manner, the quality of communication with the environment and, conse- 
quently, the quality of life for such individuáis can be greatly improved. This line of 
research, enhancing knowledge on the use of auditory interfaces as assistive technologies, 
or, more precisely, as technologies for ‘augmentative and alternative communication’ 
(AAC), is very active . 1 
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Besides these successful ways of using auditory interface technology to overeóme the 
physical restrictions of specific user groups — an understanding which seems to reflect 
Ortega y Gasset’s notion that one principie of technology is its prosthetic function 2 — a 
broader area of research has emerged that recognises and investigates the advantages of 
conveying information by auditory means for a wide variety of purposes. The idea of one 
subfield of this area, sonification, which has been a growing field since the early 1990s, 3 is 
to represent (typically numerical) data by non-speech audio events. Most challenging is the 
use of sonification as a tool for scientific data exploration, enabling researchers to listen to 
their data to search out inherent structures, trends or patterns. 

The question of ‘developing a feel’ for one’s data is a rather intangible one, and so it is 
hard to measure the success of a user interface — auditory or otherwise — at supporting it; 
yet it is also central to formulating preliminary hypotheses, and in due course attaining 
insights, concerning the data at hand. Sonification is, thus, not about alleviating physical 
restrictions, but about using additional modalities to engage with scientific data: it makes 
use of the ear in addition to, not as a surrogate for, the visual sense. In so doing it tries to 
play to the human ear’s perceptual strengths. Rather than giving a comprehensive Over¬ 
View of these strengths, we will concéntrate here on two in particular, and explain how 
these can be used in sonification design. 

To begin with, the ear can follow highly complex properties of sound, and trace múl¬ 
tiple streams of auditory events. Thus, it can be used to explore high-dimensional data 
structures or to monitor múltiple processes simultaneously. An example from music may 
help clarify this: Imagine listening to a piece of chamber music, say a string quartet. Given 
some musical training, a listener should be able to discrimínate between the different 
instruments, and to follow their paths through the piece. Translated into more scientific 
language, this means that the ear is able to filter a complex stream of sound events 
to identify single constituent streams and trace their development over time. Obviously, 
in standard (Western) scientific curricula, teaching how to listen analytically is not at all 
common practice, while teaching how to read graphs is. 

Second, regarding the pitch (or frequeney) of a sound event, the ear is extremely sensi- 
tive to minimal changes, variations and irregularities. Consider what musicians do when 
tuning their instruments, or how easy it is to detect when a piece of music speeds up or 
slows down. It has been argued, and we think rightly, that in certain contexts the human 
ear is a much more accurate ‘measuring device’ than the human eye. 4 

Thus, using sonification in a targeted and judicious manner can allow for a deeper 
understanding of complex data-sets. Conveying information by sound is understood not 
primarily as an assistive technology, but as a scientific technique. The researcher investi¬ 
gates the characteristics of the data under study by listening to their representation as 
sound events, and by manipulating aspeets of this representation interactively. It seems 
reasonable to hope that sonification may help to overeóme, to some extent, inequalities in 
the access of the visually impaired to scientific fields. But at the core of sonification lies a 
generally new approach to data; and the integration of sonification into the scientific 
toolkit would be a heuristic development the consequences of which should not be 
underestimated. 

It is interesting to consider that in a sense, sonification also aims to countervail cultural, 
as well as physical, restrictions. As we will discuss in more detail below, over the course of 
the centuries the eye has reached the status of the predominant sense in Western culture, 
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and thus also in Western Science. A conscious opening of other ways of rendering data for 
the human senses at least begins to question the valué of this predominance. 

Being aware of cultural implications and the philosophical and historical background of 
the human senses in Science and society is, at least in our perspective, crucial to an under- 
standing of the meaning of sonification. The objective of our paper is therefore to inves¬ 
tígate some of the cultural aspects of research using the human ear, before going on to 
introduce two practical examples of sonification realised using data relevant to social scien- 
tists. In this way we hope to further the use of sonification in Science, not as an alternative, 
but as a complementary tool of data analysis and exploration. First of all, however, before 
investigating aspects of the history of the human senses in Western culture, we briefly 
outline the historical roots of the attempt to capture nature in music with the help of 
numbers and elementary mathematics. 

NATURE, NUMBERS AND MUSIC 

In the history of Western Science, Pythagoras is traditionally credited with discovering the 
bridge between nature, numbers and music, by observing that subdividing strings in simple 
integer proportions (such as 2:3) created ‘harmonic’, i.e. beautiful-sounding, musical 
intervals. 5 One expression of these proportions was the Tetraktys, a triangular arrange- 
ment of the numbers one to four, symbolising, among other things, the four elements and 
the harmony of the spheres. The ensuing notion that the deep underlying beauty of Nature 
is directly related to simple numerical ratios became a central tenet of the philosophical 
school of Pythagoras, and has influenced many Western mystical and esoteric traditions 
up to the present. The notion of the harmony of musical sound based on these simple 
proportions has also deeply informed both Western tuning Systems and musical theory for 
centuries. Many composers have used the same approach to proportions for structuring 
musical form as was described for architecture by Vitruvius in antiquity and by León 
Battista Alberti in the Renaissance. 

The idea of embedding meaningful numbers in music has a long tradition too: many 
Renaissance composers encoded numbers such as birth-dates, numerological valúes of the 
ñames of their patrons (as a form of dedication), as well as more complex ideas into the 
overall scheme of notes within individual parts and sections in their scores. This esoteric 
practice was based on the Talmudic tradition of Gematria, which involves reading words 
and sentences as numbers and creating relationships of meaning between them. As Gósta 
Neuwirth has shown, it was a practice that developed to a quite amazing degree of 
subtlety. 6 However, while the idea of visually representing observed (or measured) natural 
phenomena has been a major practice of knowledge transfer (or Science) down the ages, 
transformations into music have always been of a poetical nature, not meant to be directly 
perceived; usually the encoded information is intended to remain hidden, thereby giving 
aesthetic (and possibly magical) depth to these works of art. 

Part of the inspiration for sonification for scientific purposes comes from experimental 
twentieth-century composers interested in making music that is not generated by the 
composer’s individual aesthetic preferences, but by extra-subjective factors, by ‘nature’: 
such composers have been interested in eliminating their own subjectivity to generate 
music that is not designed by a human mind. For example, John Cage (1912—92) wrote the 
composition Atlas Eclipticalis by superimposing musical staves on an atlas of the stars, and 
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by writing instructions on how to play this atlas as a graphical score. A second approach 
is exemplified by the works of Iannis Xenakis (1922—2001), who pioneered the idea of 
expressing current mathematical and philosophical thinking in music (and architecture as 
well). This typically entailed the algorithmic generation of data to give musical structures 
that would produce the desired aesthetic effects. 7 

THE EAR AS AN INSTRUMENT OF RESEARCH 

Research has, with cultural and historical variations, always relied on the human senses. 
Over the course of the centuries, however, the eye has become the dominant sense in 
research, displacing the others. Nowadays, Western culture as a whole is a visual culture: It 
is a culture of seeing, of reading, a culture of scripture and images. 8 

Science, as a part of this culture, has relied and continúes to rely on the perceptual 
capacities of the eye. Over centuries of philosophical debate, the eye emerged as the only 
sensory way to the truth, in scientific, theological and social contexts. Light was, from 
Plato’s allegory of the cave up to the Enlightenment, discursively related to truth, knowl- 
edge and humanity, both in theological and secular ideologies. 9 Medieval scholars tried to 
read the ‘Book of Nature’ in order to decipher the will of God. 10 Some hundreds of years 
later, ‘inscription devices’ are at the heart of scientific practice, transforming traces of 
matter into dots, lines or rows of numbers printed on paper that are seen to depict char- 
acteristics of the material itself. 11 This predominance of the written (or printed) in Science 
should be understood as part of a wider process of cultural development, in the course of 
which the eye emerged as the sense to be trusted when one sets out to find the truth. 
Though often challenged, the disciplined eye is seen as the only sense able to find its way 
through the promising but deceptive jungle of life, be it life in society or in nature. 12 

Notwithstanding this dominance, early scientific experiments indicate how as a measur- 
ing device the ear has sometimes been used to compénsate for a lack of adequate 
visualisation techniques. Galileo, for instance, conducted the following experiment with a 
ball falling on an inclined plañe: A small heavy ball was released down an inclined plañe so 
that as it rolled it lightly touched catgut strings that were tightened above the plañe (Fig. 1). 
Galileo noticed that every time he repeated the experiment, the sound of the strings had 
the same rhythm. 1 ’ Interestingly, he never published this methodology, instead reporting 
having measured time with water docks; he may well have anticipated scepticism towards 
an auditory proof. Though early scientific experiments like this sometimes relied on the ear 
as the ‘measuring device’, technological developments in Europe gradually furthered the 
primacy of the eye over the other senses. 

It is consistent with the notion of the predominance of the visual that the function of 
the human eye has for centuries been at the core of a variety of educated discourses. In the 
field of optics, a (scientific) field that experienced overwhelming scholarly interest from at 
least the Renaissance, people we would nowadays cali physicists, physicians, epistemolo- 
gists, philosophers, inventors and artists were taking the eye to be the central paradigm of 
their Science. From the thirteenth century up to the publication of Newton’s Opticks in 
1704, all students of optics followed the tenet formulated by Christoph Scheiner, a 
Germán Jesuit and astronomer: ‘Oculus, hoc est fundamentum opticum.’ The eye, in other 
words, was understood to be the fundamental model for explaining all optical phenomena, 
including the camera obscuraA 
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1 Inclined plañe with catgut strings and bells used by Galileo, now in the Istituto e Museo di 
Stotia della Scienza in Florence (O Photo Franca Principe, IMSS, Florence) 


It should not of course be overlooked that some practitioners — for example physicians 
and car mechanics — do indeed rely on the ear to diagnose malfunction, especially when 
dealing with unobservable areas: probably the most famous example of such auditory diag¬ 
nosis is the stethoscope (see also below). Furthermore, it would be a serious exaggeration 
to conceive of sonification as aimed at initiating a Wholesale change in our culture. Our 
basic assumption is rather that the predominance of the human eye has obscured the per- 
ceptual capabilities of the other senses. As a consequence, they are hardly ever used in 
research. We learn how to read graphical displays, and scientists’ skills are often highly 
developed and sophisticated in this area. But we do not learn how to identify structures or 
patterns in a given sequence of sounds. While graphics exist both in art and in Science, 
non-speech sound is almost exclusively restricted to the sphere of art. 

We should also be clear that we do not intend sonification as a replacement for 
visualisation. The thoughtful use of all the human senses, making good use of the percep- 
tual strengths of each, is a more complex, but ultimately more fruitful undertaking. Where 
possible, it is advisable to combine both modes of perceptual representation, visión and 
hearing, as has been done in the examples presented below. 15 

For centuries, then, technological developments fostered the predominance of the 
visual sense, and this predominance gave the direction of further research. Of course we 
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would not wish to claim unilateral causality here: the directions of technological develop- 
ment themselves rely implicidy and explicidy on theoretical, pardy metaphysical, consider- 
ations, and reinforce our faith in them. However, in the present case, the development of 
the personal Computer can be seen as the basic condition for soniflcation. To draw a graph 
by hand was quite possible millennia before the advent of the PC. But to sonify a complex 
data-set was not within the range of remotely imaginable approaches. This needed the 
combination of two interdependent historical developments: first, the development of 
Computer technology, its dissemination commercially and into scientiflc sectors other 
than those for which it was developed (for instance high-energy physics), 16 and second, 
the wide availability of PC software applications, particularly via the internet, to facilitate 
technological implementation. In addition, artists engaged in Computer music established 
laboratories whose output links advances in computers and computing with the tradition 
of extra-subjective composing. Together with research done in the area of digital audio 
synthesis and signal processing, such work forms the basis for soniflcation research. 

RESEARCH IN SONIFICATION 

The task of soniflcation research is, thus, to explore new ways of understanding scientiflc 
data. Generally speaking, understanding data depends on finding ways of simplifying 
them structurally, as in statistical analysis, or of making them accessible to our senses, for 
example as graphs or as images in scientiflc visualisations. Soniflcation takes the latter 
approach: it aims at an understanding of the inherent structures of data-sets by making 
them audible, in order to let human perception detect patterns in the data. Audio signáis 
represent elements of a data-set, and unknown structures in the respective data-set are 
mirrored by structures in the related sound events, which may well become perceptible. 

Though discussion in the field is still ongoing on how to define soniflcation, it is widely 
accepted that it involves the use of non-speech audio to display data. 17 We would argüe 
that at the core of this definition lies the constraint that soniflcation is concerned with the 
data level, and not with Information. To clarify: data become information when they are 
combined, interpreted and organised within some context which thus conveys additional 
meaning. 18 To say that soniflcation is concerned primarily with the data level means that 
the sound events are determined by the data, and the researcher listening to the sounds can 
then reason about them, interpret them, form hypotheses and thus add context, so that the 
represented data become informative. 19 In soniflcation, we listen to data in order to gather 
information. 

Soniflcation remains practically unknown in the realm of the social Sciences. This is 
despite the strengths of human hearing — for example with respect to temporal patterns, 
the detection of periodicity, and the ability to follow múltiple parallel streams of auditory 
events simultaneously — being well known. In the natural Sciences, however, a significant 
effort has been made in the past ten to fifteen years to investígate the advantages of 
soniflcation, and to explore fields of application. In 1992, on the initiative of Gregory 
Kramer, a group of researchers founded the International Community for Auditory 
Display (ICAD), which has evolved as the main international forum for professionals deal- 
ing with or interested in soniflcation and its potential. Since then, ICAD has organised 
regular meetings and conferences, the proceedings of which are the main literature source 
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in the fleld. Though there is a great deal of activity, however, sonification still lacks a 
common methodological structure, an ‘applied theory of sonification’ to unify sonification 
research. 20 

While the field of sonification is quite diverse, a kernel of techniques has emerged over 
the past ten years. The most significant are audification, parameter mapping sonification, 
and model-based sonification approaches. 21 Audification is the simplest approach. It inter- 
prets the given data as acoustical waveforms. This works best for time-series data, where 
certain observables (e.g. temperature) are measured at regular intervals, and the resulting 
variation over time (or along some other axis) can meaningfully be interpreted directly in 
terms of variations in air pressure, i.e. sound. Interesting audification applications deal with 
seismological data on earthquakes; work has also been done with EEG and stock-market 
data. 22 

Parameter mapping sonifications take a more symbolic approach. Here, the researcher 
decides how to represent the variables. The sounds are not directly generated out of the 
data, as with audification; only certain parameters of the sound event (loudness, pitch, 
timbre, duration, for example) are determined by characteristics of the data. 2 ’ The first of 
the sonification examples presented below is a parameter mapping sonification. 

Model-based approaches use models that mediate between the data and the resulting 
sound event. The objective of such models is to communicate the meaning of the data in 
clearer and more intuitive ways than parameter mappings can. In a way, the data become 
an instrument, and the user explores the data by playing the instrument. The resulting 
sonifications often require active interaction with the sonification software, for instance by 
triggering a ‘shockwave’, as is the case with the second sonification example described 
below. 24 


SOCIOLOGICAL DATA: FIELDS OF APPLICATION 

Regardless of which particular sonification technique is applied, we find that social (or 
sociological) data in general show characteristics that make them a promising field for 
sonification. They are multidimensional, and they usually depict complex relations and 
interdependencies. This was rightly emphasised in probably the first published results 
of sonification of social data. 25 This research was continued some years later and led to a 
systematic introduction to sonification for social scientists. 26 Since January 2005, these 
efforts have been integrated in a project called ‘SonEnvir’. 27 

The SonEnvir project involves a multidisciplinary consortium whose objective is to 
develop a generic approach to sonification for use in a wide variety of scientific contexts. 28 
The approach is being implemented in a sonification software environment available as 
open source for the two most common operating Systems (MacOS and Microsoft 
Windows). The examples we will present here have been conceived as test cases to help 
understand both the expectations of the target users and the heuristic valué sonification 
has to offer for their purposes. But where are the potential fields of application for 
sonification within sociology, or within the social Sciences in general? As the examples will 
indicate, the direction we consider most promising is the application of sonification to data 
capture of historical or geographical sequences. 

Sound is a time-bound phenomenon; it only exists in time, and it takes time to perceive 
sounds and their interrelations. In short, time is an intrinsic quality of sound that has to be 
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taken into account when using sound for data display. We consider this a key advantage of 
sonification. We would argüe that as a result of its inherent temporal character, sound is an 
excellent carrier of sequential information. When this time factor is not just accepted as a 
condition of auditory display, but systematically used in sonifying data, then sequential 
information can be conveyed by mapping the sequences on the implicit time axis of the 
sonification. It is for this reason that we see a major field of application of sonification in 
questions concerned with historical or geographical sequences. 

Many research areas within the social Sciences are in fact concerned with events or 
actions in their temporal context. Indeed this branch of social research is currently expand- 
ing, perhaps due to the growing importance of developmental questions and questions 
of social change. Further, the amount of social data is continuously increasing, in both a 
global sense (consider the widespread use of UN social indicators) and a historical sense 
(more statistical agencies have gathered more sophisticated and extensive data over a 
longer period). Sequence analysis, the ñeld methodologically concerned with these kinds of 
questions, applies well-established methodologies (event history analysis for example) and 
techniques to model causal relations over time. 29 Without going into detail here, all these 
analysis techniques, however advanced they may be, rely on a scenario where the 
researcher already knows where patterns are to be found in a given data-set. Comparable 
with other methods of data analysis, sequence analysis methods need to be based on an 
exploratory phase. 

PRACTICAL EXAMPLES OF SONIFICATION 

Sonification is a very good tool for exploring data-sets that represent sequential informa¬ 
tion. To ¿Ilústrate this, we introduce two sonification designs implemented in the SonEnvir 
project mentioned above. The first is concerned with time sequence data, and the second 
investigates the use of sonification for displaying geographical sequences. 

Sonification of time sequences 

The first source of sequential data sonified within SonEnvir was a project concerned with 
the development of a toilet System that meets the needs of older people and those with 
physical disabilities. 3 " In the final stage of the project, a prototype was installed at a day 
centre for patients with múltiple sclerosis in Vienna. There, the System was tested in a 
setting as cióse to real life as possible. For ethical reasons, direct observation of the user’s 
interaction with the toilet System was impossible. Thus, the solé data source showing the 
interaction of users with the equipment in this final stage was log files that were continu¬ 
ously produced by a Computer mounted near the toilet. This Computer logged the status of 
several sensors on the adjustable toilet prototype: the height and tilt of the toilet seat were 
registered, as were the status of the six buttons on a remóte control (tilt up/down, height 
up/down, ílush, alarm), and other more technical measures. 

We decided to work with this project because, from the perspective of the researchers, 
the log data were hard to deal with. It was our impression that a standard graphical display 
did not allow for a clear or intuitive representation of time sequences, especially when 
dealing with a huge number of data files. As the Computer logged about two hundred and 
forty hours in total at a resolution of a tenth of a second, a file containing a single hour’s 
data comprised about thirty-six thousand entries. 
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The situation the researchers found themselves in when beginning data analysis after 
this ‘real-life’ testing phase was that they wanted to relate actions they had not been able to 
observe direcdy to knowledge derived from examining users’ interactions in laboratory 
tests. Thus, a reconstruction from the log data of the unobserved users’ interactions with 
the equipment was required. 31 The researchers wanted to know what was going on in an 
area they could not observe visually, because they needed this information to evalúate the 
usefulness of the prototype. With some restrictions — above all, the sounds constituting 
the sonification are generated artificially; further, real-time analysis was never intended — 
this situation resembles the medical practice of stethoscopy. The stethoscope transports 
the sounds of human organs within the living body (the unobservable area), so that an- 
other person interested in their status can listen to them. The findings of this procedure 
are then evaluated by relating them to existing knowledge of (post-mortem) anatomy. By 
analogy, we listened to auditory indicators of events in the prototype toilet System, in order 
to be able to relate our findings to knowledge derived from laboratory tests. 

These data provide a good example for investigating the strengths of sonification above 
all because they record time sequences with high temporal resolution. Superficial screening 
of a data-set with these dimensions would be scarcely possible with a graphical display. 
Note that Fig. 2 shows only about ten minutes of data: if one wanted to display for 
example eight hours graphically, one would need to produce forty-eight similar graphs; 
using sonification, by contrast, the researcher could screen those eight hours in about five 
minutes. 

The sonification technique chosen in this case was parameter mapping. A specific 
sound event was assigned to each variable of interest. In total, we ended up with nine 
variables plus time as the basic dimensión. After some weeks of auditory exploration, we 
found that identifying interesting sequences is much easier with an auditory representation 
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2 Screenshot of graph showing about ten minutes of logged data from the ‘friendly rest room’ 
project 
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of the variables: with the given data, the human ear is capable of tracking the sequence of 
events even when accelerated by a factor of a hundred — in other words one is able to 
perceive a sequential order spread over a hundred seconds even when it is compressed 
into one second. 

Further, sonification provides a more dynamic representation of the events’ sequential 
order. The intrinsic time factor of the auditory representation helps the researcher to 
comprehend the sequential ordering much better than standard, static graphical displays. It 
reproduces a temporal phenomenon in its natural dimensión, time, not in space, as the 
static graph does. While dynamic graphical displays would also offer time-to-time repre¬ 
sentation, following several parameters and their timing patterns visually is much more 
difñcult. 

The conclusión we draw from this sonification example is that we were able to achieve 
what we set out to do: we were able to investígate the users’ interactions with the equip- 
ment as indicated by the logged sensor data. Though it can be argued that data within 
the social Sciences are seldom of such high temporal resolution, sonification offers a 
promising exploratory method for sequence data even when the sequences are not so 
continuously registered as they were in the example described. 

Sonification of geographical sequences 

The aim of this second sonification was to provide a representation of election results 
from the 2005 Styrian provincial parliamentary election, as an easily accessible and topical 
example of geographically distributed social data. 

Styria is one of the nine federal States in Austria, consisting of 542 communities in 
seventeen districts, and with a population of about 1.2 million. In autumn 2005, more than 
seven hundred thousand Styrians elected their political representatives. Besides the fact 
that the result of this election was in some sense remarkable — the ruling conservative party 
ÓVP (the Ósterreichische Volkspartei, or Austrian People’s Party) was defeated for the 
first time since 1945 by the Social Democratic Party, SPO (Sozialdemokratische Partei 
Osterreichs) — our interest focused on the attempt to display social data, first, in their 
geographical distribution and, second, at a spatial resolution greater than usual. We wanted 
to design a sonification that would display spatial differences and similarities in the election 
results among neighbouring communities. The technique we chose was a model-based 
sonification approach, initially developed at the University of Bielefeld, Germany, by 
Thomas Hermann and colleagues. 

The mental model behind our approach to the election data was that of a journey 
through Styria. A journey can be defined as the transformation of a spatial distribution 
into a temporal distribution. A traveller starting at community X passes first through the 
communities neighbouring X, and the longer the journey, the greater is the distance from 
community X. Henee, in this sonification, the distances between communities are mapped 
onto the time axis. 

The communities are displayed in a two-dimensional window on a Computer screen (see 
Fig. 3). For each community, the coordinates of the community’s administrative offices 
were determined and used as the geographical reference point for the respective commu¬ 
nity. The distances as well as the angles within the data thus correspond to the real 
distances and angles between the communities’ administrative offices. 
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3 Screenshot of the graphical user interface for the Styrian election sonification 


The sonification is interactive in the sense that this window can be understood as a 
musical instrument. Clicking the mouse anywhere within it initiates a circular wave that 
spreads in the two-dimensional space. The movement of this wave is shown in the 
window by an expanding red circle. When reaching a data point, this point begins to sound 
in a way that reflects its data properties. In our case, these data properties are the election 
results within each community. The researcher can select one particular party at a time to 
which to listen. Then, the pitch of the tone represents the percentage the respective party 
achieved at the election among a given community’s population: the higher the tone, the 
higher the percentage. Furthermore, the researcher can select the direction in which to 
travel. In the graph, this direction is West, indicated by the line within the circular wave. 
The line begins at the point where the researcher has initiated the wave. This line is used 
to support the perception of an aspect of the geographical distribution. 

The sonification software was designed for a ring of twelve speakers surrounding the 
listener, but it can be used with more common equipment too. In a standard stereo 
speaker setting (or when using headphones), the communities are mixed to virtual spatial 
positions between the channels (‘panned’) according to their angle from the main axis: 
those located straight in front are represented by sound events panned to the centre, those 
at ninety degrees left of the line by sound events from the left speaker, those in between 
to appropriate intermediate positions, and similarly for the right side. 

In addition to these fixed parameters, many others, for example the duration or rever- 
beration of each sound event, are more or less freely adjustable by the researcher. The 
rationale for this adjustability is that different settings allow for a focus on different aspects 
of the data-set. We found, though without being able to generalise this finding, that the 
542 valúes for a given party can be displayed in about eight to ten seconds. In relation to 
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browsing through the tables, this is a remarkable time gain. Also, in comparison with a 
standard two-dimensional graph, the combination of sound and interactive graphical dis- 
play allows for rapid geographical localisation of data points or areas of particular interest. 

Deciding on a longer decay time for individual sound events, for instance, results in a 
smoother, more integrated sound texture, because the single constituent sound events 
overlap and merge into a broader impression. This allows for a focus on general trends in 
the data. Conversely, a shorter duration for each sound event results in individual commu- 
nities being represented more distinctly. This allows for single communities to ‘pop out’. 
In fací, our experience is that this sonification is an excellent tool for outlier analysis. It 
works rather fast at a low level of aggregation (of communities), and outliers are easily 
identified by tones that are higher than in their immediate surroundings. Note that these 
are local outliers: in an area with a typical average of say thirty per cent for one party, a 
forty-two per cent result can be heard ‘popping out’; looking at the entire data-set statisti- 
cally, this may not even count as an outlier, for example if the overall average is say forty 
per cent. 

Application of this sonification software is not restricted to election data alone. Other 
social indicators assessed at the community level (for instance unemployment rates, partici- 
pation of women in the workforce, and the like) can be included. As the pitch of the 
sound event is already used for the election result, a representation option for a further 
variable may be timbre, or spectral qualities in general. To represent such indicators in 
addition to the election results facilitates the investigation of local dependencies that might 
be hidden by higher aggregation levels or by the mathematical operations of correlation 
coefficients. This is however only work in progress, and we cannot yet report conclusive 
results. 

Finally, this software is not restricted to the geographical frontiers of Styria; it could of 
course be adapted to other geographical entities. It could be used as an exploratory tool to 
enable quick scanning of social data in their geographical distribution, at different aggrega¬ 
tion levels. As it is based on open source software, it is available on the project’s homepage 
for further development. 

SONIFICATION: PRACTICAL AND CULTURAL CHALLENGES 

In an earlier section, we claimed Western culture to be a visual one. How far-reaching the 
implications of the predominance of the eye are is a source of recurrent perplexity for 
researchers concerned with auditory displays. Standard working environments often turn 
out to be quite unusable for sonification research, simply because their design follows the 
cultural hierarchy of the senses; what is more, the traditional carrier of the symbolic 
knowledge generated by Science, paper, hardly begins to meet the requirements of commu- 
nicating sound. 

The use of sound, then, has practical implications first of all for the working environ- 
ment. When using speakers, the researcher should be seated in a room alone, since it is 
likely that room-mates will not accept being subjected to Computer generated sounds over 
an extended period. When, on the other hand, one is using headphones, one makes 
oneself more or less unavailable to other technical devices that make use of an auditory 
interface. This might be harmless when missing a phone cali, but it could become more 
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dangerous in the event of a fire alarm or other more general communication system relying 
only on the auditory channel. 

The second point we want to make is concerned with scientific communication. 32 Refer- 
ring above to Latour and Woolgar’s famous study Laboratorj Life, we have claimed already 
that inscription devices are at the core of most scientific research. As Latour and Woolgar 
put it, ‘an inscription device is any Ítem of apparatus or particular configuration of such 
Ítems which can transform a material substance into a figure or diagram which is directly 
usable by one of the members of the office’. ’ 3 Moreover, this figure or diagram is not only 
directly usable by a member of the same institution, but it is portable: it can be used in a 
paper, the best established form of scientific communication. This portability bestows the 
valué of inscriptions: they can be directly used as rhetorical resources in the attempt to 
convince scientific colleagues of the truth and legitimacy of one’s claims. 

Since sonifications cannot be included in printed papers, their diagnostic power is there- 
fore weakened. To put it in other words: the applicability of sonifications as rhetorical 
resources is significantly restricted (though ICT and online journals do increasingly allow 
for the inclusión of multimedia files and the presentation of selected sound examples). 
Again, in scientific fora where sonifications could be demonstrated directly to the audience 
(for instance conferences or workshops), the technical facilities (amplifier, speakers) are 
most often inadequate. Consequently, they do not allow for serious demonstration in a 
way that invites an audience to listen carefully. This too diminishes the rhetorical forcé of 
sonification. 

We argüe that it is partly as a result of these portability problems that sonification still 
lacks recognition. Another factor surely is that to do research in sonification requires a 
certain level of training: learning to listen carefully is fundamentally different from learning 
variance analysis. As this kind of recognition is important both for an individual’s motiva- 
tion and for raising research funds, sonification research has not yet reached the critical 
mass to establish itself as a valid and valuable complementary approach in the (social) 
scientific toolkit. 


CONCLUSION 

Reporting experiences from working with practical sonifications, we find that sonification 
offers a promising complementary approach within the social Sciences, especially in the 
exploration of sequential data. Technological developments now allow for applications the 
valué of which are rarely acknowledged in the social Sciences. The idea of interactively 
exploring data depicting social phenomena is not only fascinating, but we hope also to 
have shown that it is heuristically worthwhile. It is so, we argüe, in two respects: first, 
because sonification as a tool for exploring data has something new to offer social scien- 
tists; and second, because it might initiate a reflective discussion on the extent to which the 
perceptual capabilities of the human senses are exploited in social research. 

Nevertheless, dealing with the difficulties (or — put more positively — the challenges) 
sonification faces on its way to becoming established as a scientifically reliable way of 
exploring data will be hard. The reasons for this are deeply ingrained in the fabric of our 
culture. And sonification of itself will not effect substantial change. However, we are con- 
fident that, in time, the heuristic valué of sonification will eventually be seen to outweigh 
the effort necessary to cope with the practical and cultural difficulties described. 
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NOTES 

1. To give just a few examples: so-called screen readers assist visually impaired Computer users by ‘reading’ 
the text displayed on Computer screens — for a cridcal overview see T. Stockman: ‘The design and evalu- 
ation of auditory access to spreadsheets’, Proceedings of the 2004 International Conference on Auditory 
Display (ICAD), Sydney, Australia, 2004, www.icad.org/websiteV2.0/Conferences/ICAD2004/posters/ 
stockman.pdf; a Germán team developed a microelectronic system to provide acoustic information and 
orientation for visually impaired users in cides, see M. Kemmerling and H.-J. Schliepkorte: ‘An orientation 
and informadon system for blind people based on RF-speech beacons’, in Improving the Quality ofLife for the 
European Citizen, (ed. I. Palacencia Porrero and E. Ballabio), 275—278; 1998, Amsterdam, IOS Press; and 
Scottish researchers developed a system called ScripTalker, which uses scripted conversations to help 
non-speaking people accomplish communication tasks more effectively, see R. Dye et al.\ ‘ScripTalker — 
an AAC system incorporating scripts’, in Improving the Quality of Eife for the European Citizen, pp. 310—313. 

2. J. Ortega y Gasset: Betrachtungen über die Technik\ 1949, Stuttgart, Deutsche Verlags-Anstalt (first published 
in Spanish as Meditación de la técnica , 1939). 

3. cf. T. Stockman: ‘The design and evaluation of auditory access to spreadsheets’ (see Note 1) and 
G. Kramer (ed.): Auditory Display: Sonification, Audification and Auditory Inte faces’, 1994, Reading, MA, 
Addison-Wesley. 

4. G. Kramer: ‘An introduction to auditory display’, in Auditory Display , (ed. G. Kramer), 1—77; 1994, Read¬ 
ing, MA, Addison-Wesley. See pp. 6—15 for a comprehensive summary of the advantages and drawbacks 
of auditory displays. 

5. Since Pythagoras is reported to have travelled widely in Egypt and the East, it is also possible that this 
observation may have been imported by him from the Orient. 

6. G. Neuwirth: ‘Erzáhlung von Zahlen’, in Musik-Kon^epte 26/27: Josquin des Prés; 1982, Munich, edition 
text+kritik. 

7. I. Xenakis: Formali^ed Music: Thought andMathematics in Composition\ 1992, Hillsdale, NY, Pendragon Press is 
a major reference work in contemporary music. 

8. See the works of Vilém Flusser, especially Medienkultur. ; 1997, Frankfurt am Main, Fischer, and 
Kommunikologie\ 1998, Frankfurt am Main, Fischer. 

9. H. Blumenberg: Hóhlenausgange\ 1996, Frankfurt am Main, Suhrkamp. 

10. H. Blumenberg: Die Eesbarkeit der Welt ; 1986, Frankfurt am Main, Suhrkamp. 

11. B. Latour and S. Woolgar: Eaboratoy Eife: The Construction of Scientific Facts, 1986, Princeton, NJ, Princeton 
University Press (revised edition with an introduction by Joñas Salk and a new postscript by the authors). 

12. T. Kleinspehn: Derflüchtige Blich Sehen und Identitat in der Kultur der Neu^ef 1989, Reinbek bei Hamburg, 
Rowohlt. 

13. This story is reported in S. Drake: Galilei\ 1999, Freiburg, Herder; its meaning is reflected on in F. 
Dombois: ‘Wann hóren? Vom Forschen mit den Ohren’, in Chemie — Kultur — Geschichte: FestschriftfürHans- 
Wemer Schütt anlasslich seines 65. Geburtstages , (ed. A. Schürmann and B. Weiss), 79—92; 2002, Berlin, 
Diepholz. 

14. W. Kutschmann: Der Naturwissenschajtler und sein Kórper. Die Rolle der ‘ inneren Natur’ in der experimente lien 
Naturwissenschaft derfrühen Neu^eit, 303—309; 1986, Frankfurt am Main, Suhrkamp. 

15. Building on the definition of the aim of visualisation in R. M. Friedhoff: ‘Is visualization really necessary? 
The role of visualization in Science, engineering and medicine’, in Visualisation, 1993 , 343—346; 1993, Los 
Alamitos, CA, IEEE Computer Society, we would define perceptualisation as the attempt to substitute 
conscious thinking with preconscious perceptual competencies. 

16. S. Traweek: Beamtimes and Lifetimes: The World of High-Energy Physicistr, 1988, Cambridge, MA, Harvard 
University Press. 

17. See, for instance, S. Barrass: ‘Sonification design patterns’, Proceedings of ICAD 2003, Boston, MA, USA, 
2003, www.icad.org/websiteV2.0/Conferences/ICAD2003/paper/42%20Barrass.pdf; S. C. Peres and D. 
M. Lañe: ‘Sonification of statistical graphs’, Proceedings of ICAD 2003, Boston, MA, USA, 2003, 
www.icad.org/ web site V2.0/Conferences/IC AD2003/ paper/3 8%20Peres .pdf. 

18. E. J. S. Hovenga and W. Sermeus: ‘Data analysis methods’, in Textbook in Health Informatics: A Nursing 
Perspective , (ed. J. Mantas and A. Hasman), 113—125; 2002, Amsterdam, IOS Press. 
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19. When we say that the sound events are determined by the data, we are not ignoring the fact that there is 
a translation process where the researcher has to decide how the data influence the sound parameters. 

20. ‘Sonification report: status of the field and research agenda’, prepared for the National Science 
Foundation by members of the International Community for Auditory Display, 1997, www.icad.org/ 
websiteV2.0/References/nsf.html. 

21. A more detailed description of these, and other, techniques is included in T. Hermann: Sonification for 
Exploratory Data Analysis, PhD thesis, University of Bielefeld, Germany, 2002. 

22. For earthquake seismology, see F. Dombois: ‘Using audification in planetary seismology’, Proceedings of 
ICAD 2001, Espoo, Finland, 2001, www.acoustics.hut.fi/icad2001/proceedings/papers/dombois.pdf; 
also ‘Auditory seismology’, a webpage maintained by Florian Dombois at www.auditory-seismology.org. 
For EEG data see T. Hermann et ale ‘Sonifications for EEG data analysis’, Proceedings of ICAD 2002, 
Kyoto, Japan, 2002, www.icad.org/websiteV2.0/Conferences/ICAD2002/proceedings/22_Thomas_ 
Hermann_EEG.pdf; T. Hinterberger et ale. ‘Auditory feedback of human EEG for direct brain—Computer 
communication’, Proceedings of ICAD 2004, Sydney, Australia, 2004, www.icad.org/websiteV2.0/Con- 
ferences/ICAD2004/papers/hinterberger_etal.pdf; T. Hinterberger and G. Baier: ‘Parametric orchestral 
sonification of EEG in real time’, IEEE MultiMedia, 2005, 12, (2), 70—79. For sonification of stock- 
market data see K. V. Nesbitt and S. Barrass: ‘Evaluation of a multimodal sonification and visualization 
of depth of market stock data’, Proceedings of ICAD 2002, Kyoto, Japan, 2002, www.icad.org/ 
websiteV2.0/Conferences/ICAD2002/proceedings/51_KeithNesbitt.pdf. 

23. The Sonification Lab at the Georgia Institute of Technology has developed some easy-to-handle software 
called Sonification Sandbox which supports parameter mappings. It is freely available at http:// 
sonify.psych.gatech.edu. 

24. The advantages of model-based sonifications are described in T. Hermann: Sonification for Exploratory Data 
Analysis, pp. 70—80 (see Note 21). 

25. A. de Campo and M. Egger de Campo: ‘Sonification of social data’, Proceedings of the 1999 International 
Computer Music Conference (ICMC), Beijing, China, 1999, 281—283. 

26. C. Dayé, A. de Campo and M. Egger de Campo: ‘Sonifikationen in der sozialwissenschaftlichen 
Datenanalyse’, Angewandte Sofialforschung, 2006, 24, 41-56. See also in the same volume the reviews by 
C. Frauenberger (pp. 56—60) and A. Müller (pp. 61-64) and our response to them (p. 65). 

27. SonEnvir: Designing a Generalized Sonification Environment for Scientific Data; for further information 
see the project’s homepage at http://sonenvir.at. 

28 A. de Campo, C. Frauenberger and R. Holdrich: ‘Designing a generalized sonification environment’, Pro¬ 
ceedings of ICAD 2004, Sydney, Australia, 2004, www.icad.org/websiteV2.0/Conferences/ICAD2004/ 
pos ters / campo_etal.pdf. 

29. We base our comments on A. Abbott: ‘A primer on sequence methods’, Organfiation Science, 1990, 1, 
375—392; A. Abbott: ‘Sequence analysis: new methods for oíd ideas’, Annual Review of Sociology, 1995, 21, 
93—113; H.-P. Blossfeld, A. Hamerle and K. U. Mayer: Ereignisanalyse: Statistische Theorie und Anwendung in 
den Wirtschafts- und So fialwissenschafierr, 1986, Frankfurt am Main, Campus (English title: Event History Analy¬ 
sis)', H. P. Blossfeld and G. Rohwer: Techniques ofEvent History Modelling: New Approaches to Causal Analysis', 
1995, Mahwah, NJ, Lawrence Erlbaum Associates. 

30. For a general introduction to this project see P. Panek et ale ‘Friendly rest room project: a toilet prototype 
for improving the quality of life of oíd people and persons with disability’, in Assistive Technology — From 
Virtuality to Reality, (ed. A. Pruski and H. Knops), 8—12; 2005, Amsterdam, IOS Press. 

31. C. Dayé et ale ‘Sonification as a tool to reconstruct users’ actions in unobservable areas’, Proceedings of 
ICAD 2005, Limerick, Ireland, 2005, www.idc.ul.ie/icad2005/downloads/f66.pdf. 

32. Some considerations on this topic are included in A. de Campo et ale ‘Sonification as an interdisciplinary 
working process’, Proceedings of ICAD 2006, London, UK, 2006, www.dcs.qmul.ac.uk/icad2006/ 
proceedings/papers/f54.pdf. 

33. B. Latour and S. Woolgar: Eahoratory Life, p. 51 (see Note 11). 
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